Data processing demands continue to increase in the modern era of big data, with enterprise and data center applications demanding increasingly large data sets to be processed with minimal latency. Accordingly, an increasingly important consideration for successful application deployments is to configure database systems for optimal query performance or to provide some other operational benefit, for example higher security, user friendly data formatting, or any other benefit. On the other hand, the total cost of ownership is an important counter-balancing consideration, placing a practical upper limit to performance scaling solely by hardware upgrades.
One method of providing a performance increase or to provide other operational benefits is to introduce an alternative database system based on a different technology or engine than the original database system. For example, one alternative database system may organize data in a different fashion, such as columnar-store rather than row-store, which may provide performance benefits for certain types of data and data accesses. Another alternative database system may be designed to keep all data in-memory, allowing data to be structured and database operations to be executed in a manner that can take advantage of the assumption that all data is retrievable from relatively low-latency memory rather than high-latency disk storage.
By having the alternative database system service clients or applications that would normally be serviced by the original database system, significant performance benefits may be realized. Further, by physically positioning the alternative database system closer to the client or the application host, or even on the same host, the alternative database system can function as a front tier database cache, further improving performance by reducing transmission latencies. However, not all database operations may benefit equally when migrating to the alternative database system. In some cases, database operations may see no performance benefit or even execute more slowly than on the original database system.
Furthermore, storage constraints on the alternative database system may limit the amount of data that can be copied and cached from the original database system. The original database system may be a large multi-node system with many thousands of tables, rendering it impractical to duplicate all of the data contents from the original database system. Additionally, certain alternative database system configurations may place additional limits on the cache size. For example, in the case of an in-memory database system, the cache size is further constrained by an amount of available physical memory, which in turn may be constrained due to cost, stability, availability, and other considerations.
Since it may be impractical to duplicate all of the data from the original database system, the alternative database system should be configured with only a subsection of the database that provides the greatest performance benefit. A manual process of trial and error experimentation is often the only practical method to determine that optimal subsection. This process may extend for weeks or even months, as opportunities for comparative testing may be restricted to scheduled maintenance windows to avoid performance impacts on a production database. Thus, it may be difficult to provide an accurate estimation of any potential performance benefits within a reasonable time period. Additionally, due to the manual nature of the process, there is no promise or guarantee of optimal results, human errors may be introduced, and the process may need to be repeated again in the future to accommodate evolving data sets and data access patterns, placing a heavy and continuous management burden on the database administrator.
Based on the foregoing, there is a need for a method to facilitate the integration of an alternative database system to provide highly optimized performance benefits with minimal user intervention.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.